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LIGHT MODULATED CONFOCAL 
OPTICAL INSTRUMENTS AND METHOD 

This application is a continuation-in-part application of copending 
U.S. application Serial No. 08/138,067, field October 20, 1993. 

5 Technical Field 

This invention relates generally to optical instrumentation and more 
particularly to spatially modulated light optical instruments and a method 
which employ multiple pattern illumination. 

Background Art 

10 The basic principle that distinguishes a confocal microscope from 

other forms of light microscopy is that discrete aperture spots are 
illuminated in the object plane of the microscope from which reflected or 
fluorescent light is then relayed for observation through conjugate apertures 
in the image plane. For diffraction limited spots, this principle results in 

15 spacial resolution 1.4 times better than the optimum resolution obtainable 
by conventional light microscopy. Furthermore, this method dramatically 
reduces the interference of stray, out-of-focus light from an observed 
specimen above or below the focal plane. This technique actually permits 
optical sectioning of living tissue (with depth restrictions) and high- 

20 resolution 3-D reconstruction with automated focussing controls. It is the 
only microscopic technique that can effectively resolve individual cells in 
living tissue without staining. 
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Presently known confocal microscopes employ a sequential scanning 
technique which involves scanning each illumination and image conjugate 
aperture pair one at a time until the object (and image) has been covered 
sufficiently to produce an adequate confocal image. A basic form of the 
confocal microscope is disclosed in U.S. Patent No. 3,013,467 to Minsky. 

Scanning technologies which have been developed for known 
confocal microscopes include the mechanical translation of the specimen 
with fixed optics, but scanning has also been accomplished using a fixed 
specimen and scanning beams manipulated by special rotating aperture 
disks. The use of such rotating disks is disclosed in U.S. Patent Nos. 
4,802,748 to McCarthy et al., 5.067,805 to Corle et al., 5,099,363 to 
Lichtman and 5,162,941 to Favro et al. Although these disks include a 
plurality of apertures, only one aperture at a time is used for scanning. 
Other known confocal scanning systems have included a laser beam rastered 
with rotating mirrors to scan a specimen, or a laser beam which scans a slit 
rather than a spot. Such slit scanning increases imaging speed but slightly 
degrades resolution. 

Conventional contbcal microscopes are subject to a number of 
limitations imposed by the scanning techniques employed. In some cases, 
confocal image acquisition is too slow for certain applications and becomes 
slower as the scan line density decreases and as aperture separation 
increases. Furthermore, these parameters are preset and are not practically 
adjustable in most commercial systems. Signal-to-noise ratio (SNR) must 
be severely sacrificed to increase imaging rate. Trying to improve SNR by 
increasing illumination intensity is often limited by the susceptibility of live 
cells to photodamage or rapid fluorophore photobleaching. 
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For most conventional confocal systems, proper alignment is critical 
and may be difficult to maintain. The same physical aperture must be used 
for illumination and detection thereby limiting this technique to the study 
of reflection and epifluorescence. Therefore, transmitted light confocal 
5 imaging through a translucent slice (as for many voltage-sensitive dyes) is 
not practically possible. 

Finally, it is often difficult or impossible for the user to restrict 
illumination and imaging to any selected subfield and differendally control 
light exposure. 

10 Laser scanning systems have become widely accepted because of 

their higher intensity and diffraction limited spot size which results in 
greater resolution in the confocal image. However, they are more 
expensive than white-light systems and do not offer the selection of 
illumination wavelengths needed for the growing varieties of fluorescence 

15 indicators now available. Furthermore, laser intensity often leads to 
problems with phototoxicity and rapid photobleaching, 

Confocal imaging as now known has several significant physical 
constraints. Because only a small portion (e.g. 1%) of light from a light 
source is actually used for imaging, light efficiency is challenged in 

20 multiple point source scanning confocal imaging. The requirement of a 
high power light source to compensate for the low light efficiency in 
confocal imaging results in the generation of heat necessitating the use of 
cooling fans in light modules which generate harmftil vibration. 

There exist many fluorescent indicators which operate in the dual 

25 excitation, single emission ratio mode; for example. Fura-2 and BTC for 
calcium and BCECF for pFL Dual excitation indicators have proven to be 
problematic in ratio imaging microscopy because the two images acquired 
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with different excitation wavelengths can never be obtained simultaneously. 
Reliable ratio imaging requires that the temporal separation of the two 
images be minimized relative to the rate of change of the ligand species 
which binds to the fluorescent indicator. Products for switching between 
5 two excitation sources include filter wheels and other mechanical devices 
(shutters, oscillating filters, etc) and acousto-optics modulators or tunable 
filters. Mechanical devices are slow and may introduce vibrations which 
are detrimental to imaging. Although acousto-opticals are fast, solid state 
devices, they are extremely inefficient since they operate as diffraction 
10 gratings; only a small fraction of the incident light is available for 
excitation. 

The fimdamental limitation for image acquisition speed is the number 
of fluorescent photons detected. Rapid switching between two excitation 
sources at speeds greater than that required to produce the number of 

15 fluorescence photons needed for a given signal to noise figure of merit is 
not useful. Many of the fluorescent indicators have different overall 
fluorescence efficiencies for the ligand bound and free indicator compound. 
Thus there is a need for the ability to maximize the signal to noise 
properties of ratio imaging by providing variable dwell time for the bound 

20 and free indicator compound. Then the total number of photons needed for 
a given signal to noise figure of merit can be achieved with both images 
used to form the ratio by compensating for differing fluorescence 
efficiencies with variable excitation exposure times. 
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Disclosure of the Invention 

It is a primary object of the present invention to provide a novel and 
improved spatially light modulated confocal optical instrument which 
employs multiple pattern illumination to provide conjugate aperture patterns 
5 in the image plane, 

A further object of the present invention is to provide a novel and 
improved method for confocal microscopy which includes creating a 
plurality of complementary patterns of illumination spots and superimposing 
these patterns sequentially on a specimen. 
10 Yet another object of the present invention is to provide a novel and 

improved confocal light modulation system which can be conveniently 
combined with almost any conventional research microscope to convert the 
research microscope to a confocal microscope having unique capabilities. 
A further object of the present invention is to provide a novel and 
15 improved spatially light modulated confocal microscope designed to image 
illuminate selected subsets of an entire cell field thereby enhancing image 
rate, signal-to-noise ratio, and light collection. The device will include user 
selectable program modules so that a user can fully control the optical 
transfer function of the confocal microscope in coordination with particular 
20 optical characteristics of a specimen. 

Still another object of the present invention is to provide a novel and 
improved spatially light modulated confocal microscope wherein selected 
image subsets and entire field-by-field image acquisition can be completely 
controlled by an image processing system. 
25 Yet another object of the present invention is to provide a novel and 

improved spatially light modulated confocal microscope which provides 
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external gating and control of illumination and image acquisition giving the 
user the selection of illumination and imaging from partial view fields. 
White light, other non-coherent light sources or laser illumination may be 
selected and variable speed imaging is provided. Also, a choice may be 
5 made of reflection, fluorescence or transmission confocal imaging. 

A further object of the present invention is to provide a novel and 
improved spatially light modulated confocal microscope which provides 
confocal illumination and imaging from user selected pixel aperture sizes, 
shapes and patterns which form complementary patterns to sample the 
10 whole object plane. 

Another object of the present invention is to provide a novel and 
improved spatially light modulated confocal microscope adapted for 
software-mediated tuning of the microscope optical transfer function to 
optical specimen characteristics to provide novel forms of contrast 
15 enhancement. 

Yet a further object of the present invention is to provide a novel 
and improved spatially light modulated confocal optical instrument wherein 
collimated light is patterned through a pixel shutter array having a high 
pixel density with patterned gating ability to achieve multiple pattern 
20 illumination of a specimen. 

Another object of the present invention is to provide a novel and 
improved spatially light modulated optical instrument which employs 
pattern illumination to provide real time aberration testing for an optical 
components. 

25 Yet another object of the present invention is to provide a novel and 

improved spatially light modulated optical instrument which provides rapid 
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switching between a plurality of light sources with controlled photolysis and 
pattern generation and controlled field illumination. 

A further object of the present invention is to provide a novel and 
improved spatially localized illumination system for switching between two 
5 illumination light sources to implement fluorescence ratio imaging with 
acquisition rates independent of the illumination switching time. 

Another object of the present invention is to provide a novel and 
improved spatially light modulated confocal optical instrument including a 
light source module and a confocal module with light coupling 
10 therebetween formed by a single mode optical fiber or optical fiber bundles. 

Yet a further object of the present invention is to provide a novel 
and improved spatially light modulated confocal optical instrument wherein 
defracted light not used for imaging from a pixel shutter array is recycled 
back to a system light source to enhance light source efficiency and reduce 
15 heat dissipation at the light source. 

A still further object of the present invention is to provide a novel 
and improved spatially light modulated optical instrument using only a 
single digital micro-mirror device. 

Brief Description of the Drawings 

20 Figure 1 is a basic optical schematic of a spatially light modulated 

confocal microscope embodying the present invention; 

Figure 2 is a perspective view of a portion of an electrostatic shutter 
array for use with the confocal microscope of Figure I; 

Figure 3 is a cutaway view of a single pixel for a digital mirror 
25 device array used for the present invention; 



wo 98/28655 



PCT/US97/22969 



- 8 - 

Figure 4 is an optical diagram illustrating the operation of a digital 
mirror device with the present invention; 

Figure 5 is an optical diagram showing the combination of the 
confocal light modulation system of Figure 1 with a research microscope. 
5 Figure 6 is an optical diagram of a spatially light modulated optical 

instrument including a single digital micro-mirror device and a reflective 
imaging means; 

Figure 7 is an optical diagram of a spatially light modulated optical 
instrument including a single digital micro-mirror device and a transmissive 
10 imaging means; 

Figure 8 is an optical diagram of a dual wavelength spatially 
localized illumination system; 

Figure 9 is a diagram illustrating the operation of the optical 
instrument of Figure 8; 
15 Figure 10 is an optical diagram of a spatially light modulated optical 

instrument for lens aberration testing; and 

Figure 1 1 is an optical diagram of a spatially light modulated optical 
instrument for mirror aberration testing. 

Best Mode for Carrying Out the Invention 

20 Ordinary light microscopes are often not adequate for the 

examination of biological specimens which are generally three-dimensional, 
for the two-dimensional image of the standard microscope is a superposition 
of the in-focus and out-of-focus regions of an observed specimen. 
Therefore, confocal microscopy was developed where the specimen is 

25 illuminated in a very small area one point at a time and a detector receives 
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light only from the illuminated point. The image of the whole sample is 
obtained by scanning point to point over the sample and then reconstructing 
the total image. 

Referring now to Figure 1, the spatially light modulated confocal 
5 microscope of the present invention indicated generally at 10 is 
schematically illustrated. In this device, the sequential spot scanning of 
conventional confocal microscopes is replaced by multiple pattern 
illumination (MPI). and the single channel detector, usually a 
photomultiplier tube, used with conventional sequential spot scanning is 
10 replaced by a two dimensional detector array masked by a conjugate 
aperture pattern which conforms with the multiple pattern illumination. 
Sequential complimentary patterns of illumination spots are imaged on an 
object, 

A confocal light modulation system 12, which forms the basic 
15 component of the spatially modulated confocal microscope 10, may be 
combined with the basic optics present in most conventional research 
microscopes to convert the research microscope to a confocal microscope. 
This confocal light modulation system includes a multiple pattern aperture 
array 14 which operates under the control of a central processor unit 16 to 
20 transform illuminated light from a light source 18 into sequential 
complimentary patterns of illumination spots which are imaged on a 
specimen 20 by means of a dichroic beam splitter 22 and a microscope 
objective lens 24. In the image plane, a conjugate array detector system 26 
detects the light from the illuminated spots on the specimen which are 
25 transmitted via the beam splitter 22 and a lens or mirror 28. This array 
detector system includes a two-dimensional detector 30 which is masked by 
a conjugate aperture pattern unit 32 controlled by the central processor 16. 



wo 98/28655 



PCT/US97/22969 



- 10 . 

The conjugate aperture pattern unit can be a separate unit or can constitute 
an addressed masking section of the detector 30. Under the control of the 
central processor, the conjugate aperture pattern unit masks the detector to 
define a conjugate aperture image corresponding to the pattern of the 
5 illuminated spots on the specimen onto the detector 30 in synchrony with 
the corresponding complimentary pattern formed by the multiple pattem 
aperture array 14. Both the conjugate aperture pattem unit and the multiple 
pattern aperture array and preferably high speed, pixelated arrays having a 
high pixel density which provide patterns of very small spots. 
10 The detector provides a video image output over lines 34 to the 

central processor unit which provides video image processing and display, 
or the detector could provide an output to a conventional video image 
processing and display system of known type. 

To elaborate further, using the method and apparatus of the present 
15 invention, sequential complimentary patterns of illumination spots are 
imaged on the specimen 20. Each full complimentary set of patterns will 
illuminate an entire object field without voids. The corresponding 
conjugate aperture patterns formed in front of a two dimensional detector 
modulate the image and reject stray out-of-focus light indicated in broken 
20 lines at 36, in Figure 1 from out-of-focus planes indicated by plane 38. 
This process can yield one complete image of the entire object field after 
the storage and subsequent summing of the conjugate aperture patterns 
imaged on the detector 30, 

The multiple pattern aperture array 14 can take two forms; namely, 
25 interference patterns or patterned throughput from an aperture pattern. 
Interference patterns are complex and difficult to control, and interference 
pattem integrity will be compromised by the differences in refraction and 
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birefringence across specimens of living tissue. Therefore, a multiple 
pattern aperture array 14 which provides direct aperture throughput is 
preferred for its simplicity, versatility and ease of implementation. Such a 
multiple pattern aperture array can be formed by an array of ferroelectric 
5 liquid cr\'stal devices, by a digital mirror device, or by electrostatic 
microshutters of the type illustrated at 40 in Figure 2. At rest, the 
microshutters block light from the light source 18, but when selected 
shutters are gated from the central processor 16, they open as illustrated at 
E in Figure 2 and pass light through to the specimen 20. Electronically 

10 addressed and gated microshutter arrays of the type shown in Figure 2 are 
well known and have previously been employed in display devices. (See 
R. Vuelleumier, A-E Ferret, F. Jurrez and P. Weiss (1984), Novel 
electromechanical microshutter display device. Proc. Eurodisplay '84, 
Display Research Conference Sept. SID). 

15 Confocal imaging is possible by using two separate throughput 

aperture arrays operated in synchrony for the multiple pattern aperture array 
14 and the conjugate aperture pattern unit 32. However, this requires 
extremely careful optical alignment and timing to implement. A simpler 
alternative is to employ an optical configuration that avoids use of two 

20 synchronously timed aperture arrays. This could be achieved by passing the 
image from the specimen back to the detector through the same aperture 
array used to form the multiple pattern aperture array 14, thus using this 
single array in place of the conjugate aperture pattern unit 32. For this 
purpose, the multiple pattern aperture array 14 would be positioned between 

25 the beam splitter 22 and the lens 24 in Figure 1. However, this alternative 
requires precise optical alignment and would not provide a unit which could 
be easily attached to most conventional research microscopes. 
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The conjugate aperture pattern unit 32 and detector 30 may be 

advantageously formed electronically by a CID camera or a charge-coupled 

device detector array (CCD) which uses computer aided image processing 

to enable the detector to disregard light returns from detector pixels which 

• 

5 are not activated. A charge-coupled detector has the ability to acquire an 
entire image at once rather than by raster scanning. A complete bit map 
must be constructed of where active pixels are located on the CCD after 
calibration for each aperture pattern to be provided by the multiple pattern 
aperture array 14 under the control of the central processing unit 16. 
10 Calibration simply requires training the processing unit 16 which controls 
the CCD to disregard pixels which should not be read by placement of a 
mirror in the objective focal plane of the microscope so that each 
illumination pattern from the multiple pattern aperture array can be imaged 
directly on the CCD and stored by the central processing unit. Additional 
15 electronic and computer aided filtering after calibration may be required to 
clarity aperture boundaries and improve spatial resolution and sectioning 
capability- but once this is done, only light transmitted to the CCD for an 
associated pattern from the multiple pattern aperture array 14 is imaged, 
while light spillage to other areas of the CCD is ignored and effectively 
20 masked. This technique provides practical transmission confocal 
microscopy because the illumination subsystem does not use the same 
optical path as the imaging subsystem as is evident from Figure 1. 

For a CCD unit to work at the high speeds desired for confocal 
microscopy, the detector pixel registers for the CCD device must be read 
25 rapidly in parallel for all lines. Therefore, special CCD pixel driver 
circuitry is required to optimize the detector for high speed applications. 
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The need for the special driver circuitry required with a CCD 
detector to read lines in parallel can be eliminated by forming the detector 
30 from an active pixel sensor (APS) rather than from a CCD. The active 
pixel sensor operates in a manner similar to that of a random access 
5 memory with each pixel containing its own selection and readout 
transistors. These transistors are monolithically integrated in the pixel for 
readout selection and amplifying or buffering the pixel output signal. An 
APS pixel pattern selected by the central processor 16 receives the light 
pattern from the specimen 20 and a photogenerated charge is integrated in 

10 each active pixel under a photogatc. The selected pixels can be addressed 
simultaneously to provide output signals to an image processing system or 
to the central processing unit 16 for image processing. 

The multiple pattern aperture array 14 and, for some applications, the 
conjugate aperture pattern unit 32 may preferably be formed by a digital 

15 micro-mirror device of a type developed by Texas Instruments Inc. of 
Dallas, Te.xas, which includes a monolithic electrostatic, mirror-coated 
shutter array. These pixelated, micro-mechanical spatial light modulators 
are formed monolithically upon a silicon chip and include minor squares of 
highly reflective material suspended above the surface of silicon addressing 

20 circuitry by support pillar torsion bars. The addressing circuitry under each 
mirror pixel is a memory- cell accessed by the central processor unit 16 
which drives two electrodes under each mirror with complimentary signals. 
When a high logical level is applied to one of the electrodes, the mirror 
pixel rotates in a given direction with a total deflection of about 20^ 

25 between two mirror states, 10*" on either side of the perpendicular to the 
silicon wafer. 
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Figure 4 illustrates the basic structure of one pixel for a digital 
mirror device array. Here, a mirror coated shutter 42 is mounted in contact 
with a conducting support layer 44 by torsion hinges 46. A spacer layer 48 
separates the support layer 44 from a silicon chip 50. Each pixel of the 
5 digital mirror device array includes an address electrode 52 and a control 
or landing electrode 54 which, when activated, deflects the mirror coated 
shutter 42 around the torsion hinges 46. When the mirror coated shutter 42 
forms part of a multiple pattern aperture array mounted in the optical path 
of the illumination subsection 56 for a microscope, it may be positioned in 
10 a first position as shown in Figure 4 to direct a pixel of light along an 
optical path 58 to a specimen under observation. However, when the 
mirror coated shutter 42 is deflected from the position of Figure 4, the pixel 
of light is angled away from the specimen along an optical path 60. 

Software for the central processor unit 16 will enable a pre- 
15 programmed selection of any possible two dimensional pixel pattern within 
predetermined limits for confocal imaging. This technology provides the 
user with the flexibility to choose the pattern which offers the best 
combination of temporal and spacial resolution for a given application. The 
ability to custom make patterns can be geared for the correction of field 
20 distortion. The individual complementary images formed from each pixel 
pattern received by the detector 30 will be stored in the central processor 
unit 16 and subsequently overlaid to form one complete image frame for 
real-time imaging. 

The control software for the central processor 16 will be 
25 programmed to select stored aperture patterns designated by the user and, 
for some applications, to automatically select stored aperture patterns in 
response to sensed conditions such as the optical characteristics of a 
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specimen. The processor software may also cause the processor to make 
a specimen-dependent background correction for each image frame. This 
is accomplished by sensing with the detector 30 a background image which 
may result with all illumination pixels of the multiple pattern aperture array 
5 14 closed to block light from the light source 18. and then correcting in the 
processor the subsequent individual complimentary images fomied from 
each pixel pattern by subtracting therefrom the effect created by the 
background image. 

Figure 5 illustrates one manner in which the confocal modulation 

10 system 12 may be incorporated into the structure of a conventional research 
microscope 62. The multiple pattern aperture array 14 is inserted into the 
optical path of the illumination subsystem 56 for the microscope while the 
conjugate aperture pattern unit 32 is inserted into the optical path of the 
microscope image subsection 64. The remaining optical components shown 

15 in Figure 5 are components of the research microscope 42. 

The successful operation of a light microscope is critically dependent 
on its illumination system. Specimens are usually non-luminous and must 
therefore be observed via light transmission, reflection, or fluorescence 
which requires the help of an auxiliary system consisting of an illumination 

20 source, condenser, and field aperture stops. Reflection or fluorescence 
observations usually employ epi-illumination where the microscope 
objective also functions as the condenser lens. Basic control of the 
illumination path and profile in any of these approaches usually employs 
one of two methods known as critical illumination and Kohler illumination. 

25 Critical illumination is achieved by placing the light source behind 

the field stop and imaging the source onto the object plane of the 
microscope objective. With Kohler illumination, an auxiliary lens, located 



wo 98/28655 



PCT/US97/22969 



- 16 - 

behind the field stop, forms an image of the source in the back focal plane 
of the condenser. The object plane of the microscope objective is 
illuminated with superimposed collimated beams corresponding to each 
source point that tend to average out spatial irregularities in source 
brightness. For this reason, Kohler illumination is usually preferred over 
critical illumination. Nevertheless, Kohler illumination only approximates 
illumination uniformity and is not optimal for many applications. 

Current microscope illumination systems have evolved primarily to 
concentrate light on the specimen. External devices placed along the light 
path can manipulate properties of the light such as wavelength, polarization, 
and intensity. However, these devices often serve only one purpose and 
lack the flexibility required when combined with other microscope 
modalities. 

In accordance with the present invention, both reflective and 
transmissive confocal light modulated optical imaging systems have been 
developed wherein the conjugated illumination aperture and the detection 
aperture can be configured by double passing through a single digital 
micro-mirror device. 

Figure 6 illustrates a confocal light modulated optical imaging 
system indicated generally at 66 which uses all reflective components to 
perform 1 ; 1 imaging. This system has no chromatic aberration and less 
spherical aberration. The system 66 includes a light source module 68 
containing a light source 70 and any cooling fans required (not shown). 
The light source module is isolated from the remainder of the light 
modulated optical imaging system 66 and is connected thereto by a single 
mode optical fiber or optical fiber coupling bundles 72 which transmit light 
from the light source module while isolating the system from mechanical 
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vibrations which may be present in the light source module. For a coherent 
light source 70, a single mode optical fiber will be used for the coupling 
72, while for an incoherent white light source, an optical fiber bundle will 
be used. Light from the coupling 72 is directed onto a parabolic mirror 74 
5 and is reflected to a beam splitter 76. From the beam splitter, light is 
reflected to a first spherical mirror 78 and then to a second spherical mirror 
80 which reflects it back to the first spherical mirror. The light is then 
reflected to mirrors 82 and 84 which direct light onto a computer controlled 
digital micro-mirror device 86 of the type previously described. Mirrors 82 

10 and 84 arc not always required for directing light to the digital mirror 
device 86. Without mirrors 82 and 84 light can also be directed to the 
digital mirror device 86 by rotating the angle of the optical system fonned 
by the spherical mirrors 78 and 80. The device 86 can be an array of 864 
X 756 mirrors that can be individually tilted ± 10° dependent on the state 

15 of an associated binary memory cell. In one angle state a mirror will direct 
light down a light path 88 while in the second angle state the mirror will 
direct light to a parabolic mirror 90. 

In multiple point confocal imaging of the type attained with the 
device 86, as little as 1% of the light received from the mirror 84 is 

20 directed down the light path 88 for imaging and the rest is deflected. This 
not only results in poor illumination efficiency, but the deflected light will 
cause unwanted heat in the vicinity of device 86 that would normally 
require the use of a cooling fan which creates unacceptable mechanical 
vibrations. In accordance with the present invention, the mirror 90 is 

25 provided to receive the deflected light and direct it into an optical cable or 
cable bundle 92 that returns this light to the coupling 72. This recycling 
of the unused difracted light both raises the light efficiency of the system 
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while reducing heat dissipation in the area of the device 86 so that no 
cooling fan is required. 

The light pattern from the digital micro-mirror device 86 is directed 
: along the light path 88 to a third spherical mirror 92 and is reflected 
5 ' thereby to a fourth spherical mirror 94. The pattern is then reflected from 
the spherical mirror 94 back to the spherical mirror 92 which reflects the 
pattern onto an image plane 96. The image plane 96 represents the image 
plane of a microscope objective which relays light to a specimen under 
observation. 

10 The image from the specimen is directed back to the image plane 96 

and passes back along the same path followed by light from the light 
• module 68. It is reflected back by the micro-mirror device 86 and when it 
reaches the beam splitter 76, it is directed onto the image plane 98 of a 
detector such as the detector 30. By double passing a single digital micro- 

15 mirror device with both the illumination light and the pattern image from 
the specimens, alignment problems which can arise from the use of a 
separate detector mask are eliminated. Axial resolution is also improved in 
comparison with the axial resolution in the ideal single pinhole confocal 
imaging case. The micro-mirror device reflects a light image to the 

20 detector which is in a pattern corresponding to the pattern of illumination 
spots initially created by the micro-mirror device and rejects light outside 
said pattern. 

Computer simulations have shown that the intensity contrast ratio of 
the micro-mirror device on pixel to off pixel in the range between 20 and 
25 10,000 can improve the axial resolution in comparison with the axial 
resolution of the ideal single pinhole confocal imaging case. 
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Figure 7 illustrates a confocal light modulated optical imaging 
system 100 with a transmissive imaging means where components having 
a similar structure and operation to those shown in Figure 6 will be 
identified with like reference numerals. Here a combination of lenses and 
5 mirrors illuminate the digital micro-mirror device to generate the confocal 
scanning image. Light from the light module 68 and difracted light from 
the optical cable 92 is passed by the optical coupling 72 and the beam 
splitter 76 to a first lens 102 which transfers the light on a mirror 104. The 
mirror 104 then reflects the light onto the digital micro-mirror device 86, 

10 and the deflected light from the micro-mirror device is reflected by a mirror 
106 through a lens 108 into the optical cable 92. The light pattern from the 
micro-mirror device is transmitted along the light path 88 and is transferred 
by a lens 1 10 on a mirror 112. The mirror 112 reflects the light pattern to 
a mirror 1 14 which in turn reflects the light pattern to a mirror 1 16. Mirror 

15 116 reflects the light pattern onto a microscope image plane 118, which in 
the case of a transmissive confocal light modulated imaging system, is the 
image plane of the microscope illumination condenser on the side of the 
specimen which is opposite to the microscope objective. As in the case of 
the reflective confocal imaging module, the light pattem from the specimen 

20 passes from the image plane 1 18 back over the same optical path taken by 
the light from the light module to the beam splitter 76. The beam splitter 
directs this light to a mirror 120 which then reflects the light pattern onto 
the image plane of a detector 122. 

The ability to alternate rapidly between uvo illumination sources 

25 extends the capabilities of conventional light microscopes. In theory, any 
interaction between light and matter which alters a property of the incident 
light can be used to generate an optical image. For example, wavelength. 
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circular dichroism, and polarization are all properties of light that can be 
varied for image formation. Switching between left and right circularly 
polarized or linearly polarized light can be used to generate polarization 
difference images. The use of two fluorescent indicators, each operating in 
5 the single excitation mode, can be achieved by switching between two 
excitation wavelengths. With variable dwell times, the user can optimize 
the collection of data derived from two sources having different time scales. 
For example, a slowly changing pH signal could be monitored in the 
presence of a rapidly changing calcium signal using SNARP and Indo-I in 

10 hypoxic brain slices. 

In accordance with the present invention, a spatially localized 
illumination system has been developed for operation in a dual wavelength 
mode. In this mode, fluorescence ratio imaging will be implemented with 
acquisition rates independent of the illumination switching time between 

15 two system light sources. The system provides for controlled photolysis 
and pattern generation, rapid switching between two light sources and 
controlled field illumination. The system further offers the ability to 
maximize the signal to noise properties of ratio imaging by providing 
variable dwell time for the bound and free indicator compound. With this 
20 device, the total number of photons needed for a given signal to noise 
figure of merit can be achieved with both images used to form the ratio by 
compensation for differing fluorescence efficiencies with variable excitation 
exposure times. 

Referring now to Figures 8 and 9, a novel microscope illumination 
25 modulation system (MIMS) indicated generally at 124 is illustrated. MIMS 
124 is designed to operate in a critical illumination mode and in an 
alternating dual source illumination mode. In the critical illumination 
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mode, MIMS will provide computer-controlled and spatially localized 
illumination of an object plane over pixels approaching the diffraction limit. 
For this mode, control paradigms for spatial photolysis of "caged" 
compounds, pattern generation for fluorescence recovery after 
5 photobleaching and photoactivation of fluorescence can be provided. 

In the dual-source illumination mode, MIMS provides submillisecond 
switching betv/een two light sources. Thus MIMS is a multipurpose 
illumination attachment that can be used in either the epi-illumination or 
condenser port of an optical microscope such as that shown in Figure 5. 

10 It is also intended to be used as a light source for the confocal modulation 
systems shown in Figures L 4, 6 and 7. 

The microscope illumination modulation system 124 includes first 
and second light sources 126 and 128 which provide light beams that are 
transferred respectively by lenses 130 and 132 and mirrors 134 and 136 

15 onto a digital micro-mirror device 140 which is controlled by a processor 
138 or other sequencing control unit. Figure 9 illustrates the basic optical 
diagram for one digital micro-mirror pixel 142. In sequence A, the mirror 
pixel 142 is inclined ten degrees to the right to direct a light beam from the 
light source 126 down an optical path 144 while diverting the light beam 

20 from the light source 128 away from the optical path. This diverted beam 
can be picked up by a light trap 146, such as the parabolic mirror 90, which 
feeds light through a fiber optic cable back to the source. 

In sequence B of Figure 9, the mirror pixel 112 is inclined ten 
degrees to the left to direct a light beam from the light source 128 down the 

25 optical path 144 wliile diverting the light beam from the light source 126 
away from the optical path. The lens and mirror combinations 130 and 134 
and 132 and 136 are positioned and designed to direct the light beams from 
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the light sources 126 and 128 at angles to the digital micro-mirror device 
140 which enable the selected mirror pixels of the digital micro-mirror 
device to operate in the manner illustrated in Figure 9. The digital micro- 
mirror device has an array of 864 x 756 mirror pixels that can be tilted plus 
or minus ten degrees dependent upon the zero or one state of an associated 
binary memory cell. The mirrors can switch between these two angle states 
in only 30 microseconds. 

With two light sources 126 and 128 of different light wavelengths, 
complementary patterns of two wavelengths can be created. 

When the microscope illumination modulation system 124 is used in 
the critical illumination mode, an image of the digital micro-mirror device 
will be generated in the object plane of the microscope. Using a standard 
computer interface, (mouse, touchpad, pen, etc.) the operator will be able 
to outline an area or areas on the computer display for subsequent 
illumination by one or more illumination sources coupled to the MIMS 124. 
When the external light sources are triggered, only the defined area will be 
illuminated. Neurotransmitters could be released from caged precursors in 
dendritic regions. Specific chromosomes could be targeted for ablation in 
chromosome surgery. 

In the altemating dual-source illumination mode of the MIMS 124, 
the digital micro-mirror device will be used to switch between the two 
external light sources. The operator can use either the processor 138 to 
control the switching rate and duty cycle or some other external 
synchronization source. This control will allow optimal illumination of 
dual-excitation single-emission fluorescent compounds such as the calcium 
indicator Fura-2. 
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Finally, in the uniform illumination mode, Kohler illumination will 
be used to illuminate the surface of the digital micro-mirror device. Kohler 
illumination is a method of adjusting the illumination system of a 
microscope which offers several advantages including maximizing contrast, 
5 and lateral and axial resolution. Maximum performance is achieved by 
illuminating the specimens with a converging set of plane wave fronts, each 
arising from an image of the source in the condenser aperture and conjugate 
with the back focal plane of the objective. Optimum performance is only 
achieved when the condenser aperture is uniformly illuminated. This is 

10 difficult to achieve with concentrated arc-lamps which are highly 
nonuniform. Often, luminous density is sacrificed by over-filling the 
aperture in order to approximate uniform illumination. MIMS 124 has the 
ability to control the illumination of the condenser aperture using the digital 
micro-mirror device to create a time-varying mask. This feature v/ill 

15 enhance the use of the microscope in any of the modes of microscopy 
which rely on uniform illumination to generate image contrast. These 
modes include phase contrast, interference contrast, differential interference 
contrast, modulation contrast, darkfield, absorption, reflection, polarization 
and fluorescence. 

20 For some applications, it would be possible to direct selected pixels 

of light from both light sources 126 and 128 simultaneously down the 
optical path 144. To accomplish this, the mirrors 134 and 136 would direct 
light from a respective light source on an area of the micro-mirror device 
140 v/hich does not receive light from the remaining light source and the 

25 pixels in each area would be separately controlled by the processor 138. 
For dual color light projection, the light sources 126 and 128 would provide 
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light of different colors. Other differing light characteristics could be 
separately provided by these two light sources and simultaneously projected. 

Figure 10 illustrates a spatially light modulated optical lens testing 
instrument and method for real time lens aberration testing. In its simplest 
form, the lens testing instrument indicated generally at 148 includes a light 
source 150 which projects light on a digital micro-mirror device 152 that 
is controlled by a processor 154. An optical system, indicated for 
illustrative purposes as a mirror 156, directs light from the digital micro- 
mirror device 152 along an optical path through a lens 158 under test. 
Each pixel mirror of the digital micro-mirror device can constitute a 
separate beam source under the control of the processor 154, although for 
illustrative purposes, only two of such beams are shown in Figure 10. Each 
beam passes through the test lens and contacts an optical detector 160 at a 
definite location. The position of each pixel beam on the detector is 
indicative of the condition of the test lens 158, for with a properly 
configured lens, each beam will contact the detector at a defined contact 
point programmed in the processor 154. Deviation of the beam from this 
defined contact point indicated aberration in the lens in the area through 
which the beam passes, and detector position data indicative of this 
deviation is fed to the processor 154 where it is compared with the stored 
position data to obtain deviation data values. These deviation data values 
can be used in a printout or display to indicate where and to what degree 
lens correction is required or can be directed as control data for a lens 
grinder or other lens formation device to directly control correction of the 
test lens 158. 

Obviously, in practice the- lens testing instrument can include optical 
components in addition to those shown diagrammatically in Figure 10. For 
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example, a reflective lens test system can employ the complete confocal 
light modulated optical system 66 of Figure 6 with the exception that the 
lens 158 under test would be placed in the position of the image plane 96 
and a mirror 162, shown in broken lines in Figure 6 would be placed 
5 behind the lens 158 to reflect the pixel beams back to the spherical mirror 
92 and back along the light path to the image plane 98 of the detector 160. 

As shown in Figure 1 1, optical components other than lenses can be 
tested using a testing instrument similar to that of Figure 10. Here the 
testing instrument 148 is used to test a mirror 162 rather than a lens, and 

10 employs a beam splitter 164 to direct light reflected from the test mirror to 
the optical detector 160. Thus, the optical component test system 148 uses 
each pixel in a spatial light modulator (e.g., micro-mirror device 152) to 
generate a real-time ray tracing. The ray tracings pass through a test lens 
or are reflected by a test mirror and finally reach a two-dimensional 

15 detector 160 (e.g., CID or CCD) which is conjugated to the spatial light 
modulator plane in object-image relation. The position and the distribution 
of the ray tracings on the detector plane indicate the aberration situation of 
the test optical component. The signal processing means implemented in 
central processor unit (CPU) 154 retrieves the structural information of the 

20 test optical component after it collects the position and the distribution of 
the ray tracings on the detector plane. 

At the present time, almost all of the commercial confocal 
microscope systems have very low photon signals, e.g., from 20 to 100 
photons. Such a low number of photons has a very small signal-to-noise 

25 ratio (SNR) because of quantum noise. A way to improve the SNR of a 
confocal image can comprise the following steps. In the first step, use a 
large illumination aperture and a large detection aperture to get a confocal 
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image which has a high SNR and low axial resolution. In the second step, 
use another different large illumination aperture and a large detection 
aperture to get another confocal image which has a high SNR and low axial 
resolution. In the third step, use the difference of said confocal images to 
5 get the final confocal image which has both high SNR and high axial 
resolution. 

In microscopic imaging, different imaging modes, e.g., confocal 
scanning imaging mode, wide-field fluorescence imaging mode, bright field 
imaging mode, dark field imaging mode, contrast (phase contrast, 
10 modulation contrast, interference contrast, and polarization contrast) 
imaging modes, will get different information. To spatially correlate the 
information from different imaging is very helpful for analyzing a sample. 

By using discrete and digital illumination and discrete and digital 
detection, we can easily realize such correlation. The system comprises the 

15 following elements and relationships; namely, a group of discrete, analog 
or digital detection elements sampling the detection plane, an object plane, 
which samples a slice of sample, substantially conjugating with both said 
illumination plane and said detection plane in object-image relationships, 
a set of complimentary confocal patterns formed by said illumination 

20 elements and said detection elements generating a confocal imaging mode. 
By setting all said illumination elements at the same time to the same level 
or different levels, e.g., turning on all DMD pixels at the same time, the 
illumination plane becomes a uniform reflection plane, or a uniform 
transmission plane, thus switching to other imaging modes. Said spatial 

25 correlation is realized by a signal processing means which correlates the 
signals from each detection element at the same address in different 
imaging modes. 
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Industrial Applicability 

The spatially light modulated confocal optical instruments and 
method of the present invention provides conjugate aperture image 
formation to counter the deficiencies of known devices such as scanning 
5 confocal microscopes. The confocal light modulation system may be used 
to convert a conventional research microscope into a confocal microscope 
having enhanced capabilities. Since the unit has no moving parts and is 
vibration free, it is suitable for sensitive micro-manipulation and 
intracellular recording techniques common in neurophysiological studies. 

10 The high speed imaging capability of the unit renders it suitable for use to 
follow dendritic and synaptic changes in neurons on the millisecond scale 
or to measure kinetics in response to drugs or fluorescent indicators for 
intracellular pH, calcium, sodium, etc. The spatially light modulated 
confocal optical instrument is particularly well adapted for use in the high 

15 speed inspection of silicon wafers for quality control purposes in integrated 
circuit production, and for testing high quality lenses. 
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We claim; 

1. A confocal optical imaging system for imaging a specimen 
. comprising: 

a light source for providing light to the specimen, 
5 optical means to direct light from said light source along an optical 

path to said specimen, 

pattern array means positioned in said optical path between said light 
source and said specimen which operates to transform light from said light 
source into a plurality of simultaneously formed illumination spots arranged 
10 in a pattern and to block light outside said pattern, said optical means 
operating to image said pattern of illumination spots simultaneously on said 
specimen, 

image detector means, 

said optical means operating to direct light from said pattern of 
15 illumination spots on said specimen back along said optical path to said 
image detector means, 

said pattern array means operating to mask said image detector 
means to cause said image detector means to receive a light image in a 
pattern corresponding to the pattern of illumination spots and to reject light 
20 outside said pattern. 

2. The confocal optical imaging system of claim 1 wherein said 
pattern array means is operative in response to control signals to provide 
sequential patterns of illumination spots, and a control means is provided 
which is connected to said pattern array means, said control means 
25 operating to provide control signals to said pattern array means to cause 
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said pattern array means to provide sequential complimentary patterns of 
illumination spots. 

3. The confocal optical imaging system of claim 2 wherein said 
pattern array means includes a digital mirror device having a mirror coated 

5 shutter array. 

4. The confocal optical imaging system of claim 3 wherein said 
image detector means provides image output signals from each pattern of 
light received by said image detector means, said control means including 
storage means to receive and store said image output signals. 

5. The confocal optical imaging system of claim 4 which 
includes image processing means, said image detector means operating to 
provide image output signals from each pattern of light received by said 
image detector means to said image processing means, said image 
processing means operating to store said image output signals from 
sequential patterns of illumination spots and to subsequently combine a 
plurality of said stored image output signals to form a complete image 
frame. 

6. The confocal optical imaging system of claim 5 wherein said 
image detector means includes a charge coupled detector device. 

20 7. The confocal imaging system of claim 1 which includes light 

collector means mounted to receive light blocked by said pattern array 



10 



15 
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means and light transmission means to transmit light from said light 
collector means to said light source. 

8. The confocal imaging system of claim 7 wherein said light 
source includes a light generator and a flexible fiber optic light conductor 
for conducting light from said light generator to said optical means. 

9. A confocal optical imaging system for imaging a specimen 
comprising: 

a light source for providing light to the specimen, 
optical means to direct light from said light source along an 
illumination path to said specimen, 

pattern array means positioned in said illumination path between said 
light source and said specimen, said pattern array means operating to 
transform light from said light source into a plurality of simultaneously 
formed illumination spots arranged in a pattern and to deflect light outside 
said pattern, 

said optical means imaging said pattern of illumination spots 
simultaneously on said specimen, 

light collector means mounted to receive light deflected by said 
pattern array means and light transmission means to transmit light from said 
light collector means to said light source. 

10. The confocal imaging system of claim 9 wherein said light 
source includes a light generator and a flexible fiber optic light conductor 
for conducting light from said light generator to said optical means. 
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1 1 . The confocal optical imaging system of claim 10 wherein said 
partem array means is operative in response to control signals to provide 
sequential patterns of illumination spots, and a control means is provided 
which is connected to said pattern array means, said control means 

5 operating to provide control signals to said pattern array means to cause 
said pattern array means to provide sequential complimentary patterns of 
illumination spots. 

12. The confocal optical imaging system of claim 10 wherein said 
pattern array means includes a digital mirror device having a mirror coaled 

10 shutter array. 

13. A light pattern generating system for directing a light pattern 
down an optical path comprising a digital mirror device having a mirror 
coated shutter array which is operative in response to control signals, said 
mirror coated shutter array including a plurality of mirror coated shutters 

15 each movable in response to said control signals to move to either a first 
or a second angular position relative to said optical path, 

a first light source positioned relative to said mirror coated shutter 
array to direct light on said mirror coated shutters in said first angular 
position, said mirror coated shutters operating in said tlrst angular position 

20 to direct light from said first light source down said optical path and in said 
second angular position to reflect light from said first light source away 
from said optical path, 

a second light source positioned relative to said mirror coated shutter 
array to direct light on said mirror coated shutters in said second angular 

25 position, said mirror coated shutters operating in said second angular 
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position to direct light from said second light source down said optical path 
and in said first angular position to reflect light from said second light 
source away from said optical path, and 

control means connected to said digital mirror device to provide 
5 control signals thereto, 

14. The light pattern generating system of claim 13 which 
includes light collection means to collect light reflected from said mirror 
coated shutter array and light transmission means connected to transmit 
light from said light collection means to said first and second light sources. 

15. The light pattern generating system of claim 13 wherein said 
mirror coated shutter array is operative to provide patterns of illumination 
spots from light received from said first light source when a pattern of said 
mirror coated shutters are in said first angular position and a pattem of 
illumination spots from light received from said second light source when 
a pattem of said mirror coated shutters are in said second angular position, 

said control means operating to provide control signals to said mirror 
coated shutter array to control the pattern of illumination spots transmitted 
down said optical path from said first and second light sources. 

16. A method for real time aberration testing of a lens which 
20 includes 

placing said lens in an optical path for a light source, 
placing a digital micromirror device having a two-dimensional array 
of pixels in said optical path between said light source and said lens, 



10 
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switching pixels of said digital micromirror device to create one or 
more beams of light which pass through said lens, and 

detecting the beams of light which pass through said lens to 
determine the position thereof. 

5 17. A method for real time aberration testing of a mirror which 

includes 

placing said mirror in an optical path for a light source, 

placing a digital micromirror device having a two-dimensional array 

of pixels in said optical path between said light source and said lens, 
10 switching pixels of said digital micromirror device to create one or 

more beams of light which are directed onto said mirror and are reflected 

thereby, and 

detecting the reflected beams of light to determine the position 
thereof. 
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